Erasure Coding in Windows Azure Storage

نویسندگان

  • Cheng Huang
  • Huseyin Simitci
  • Yikang Xu
  • Aaron Ogus
  • Brad Calder
  • Parikshit Gopalan
  • Jin Li
  • Sergey Yekhanin
چکیده

Windows Azure Storage (WAS) is a cloud storage system that provides customers the ability to store seemingly limitless amounts of data for any duration of time. WAS customers have access to their data from anywhere, at any time, and only pay for what they use and store. To provide durability for that data and to keep the cost of storage low, WAS uses erasure coding. In this paper we introduce a new set of codes for erasure coding called Local Reconstruction Codes (LRC). LRC reduces the number of erasure coding fragments that need to be read when reconstructing data fragments that are offline, while still keeping the storage overhead low. The important benefits of LRC are that it reduces the bandwidth and I/Os required for repair reads over prior codes, while still allowing a significant reduction in storage overhead. We describe how LRC is used in WAS to provide low overhead durable storage with consistently low read latencies.

برای دانلود متن کامل این مقاله و بیش از 32 میلیون مقاله دیگر ابتدا ثبت نام کنید

ثبت نام

اگر عضو سایت هستید لطفا وارد حساب کاربری خود شوید

منابع مشابه

Giza: Erasure Coding Objects across Global Data Centers

Microsoft Azure Storage is a global cloud storage system with a footprint in 38 geographic regions. To protect customer data against catastrophic data center failures, it optionally replicates data to secondary DCs hundreds of miles away. Using Microsoft OneDrive as an example, this paper illustrates the characteristics of typical cloud storage workloads and the opportunity to lower storage cos...

متن کامل

CodePlugin: Plugging Deduplication into Erasure Coding for Cloud Storage

Cloud storage systems play a key role in many cloud services. To tolerate multiple simultaneous disk failures and reduce the storage overhead, today cloud storage systems often employ erasure coding schemes. To simplify implementations, existing systems, such as Microsoft Azure and EMCAtmos, only support file appending operations. However, this feature leads to a nontrivial and increasing porti...

متن کامل

On the Average Locality of Locally Repairable Codes

A linear block code with dimension k, length n, and minimum distance d is called a locally repairable code (LRC) with locality r if it can retrieve any coded symbol by at most r other coded symbols. LRCs have been recently proposed and used in practice in distributed storage systems (DSSs) such as Windows Azure storage and Facebook HDFS-RAID. Theoretical bounds on the maximum locality of LRCs (...

متن کامل

Applied Erasure Coding in Networks and Distributed Storage

The amount of digital data is rapidly growing. There is an increasing use of a wide range of computer systems, from mobile devices to largescale data centers, and important for reliable operation of all computer systems is mitigating the occurrence and the impact of errors in digital data. The demand for new ultra-fast and highly reliable coding techniques for data at rest and for data in trans...

متن کامل

Windows Azure Platform: an Era for Cloud Computing

Windows Azure platform is the Microsoft implementation of cloud computing. This paper covers detailed introduction to Windows Azure Platform. Windows Azure provides resources and services for consumers. The next part describes the five main components of Windows Azure: Hardware is abstracted and exposed as compute resources. Physical storage is abstracted as storage resources and exposed throug...

متن کامل

ذخیره در منابع من


  با ذخیره ی این منبع در منابع من، دسترسی به آن را برای استفاده های بعدی آسان تر کنید

برای دانلود متن کامل این مقاله و بیش از 32 میلیون مقاله دیگر ابتدا ثبت نام کنید

ثبت نام

اگر عضو سایت هستید لطفا وارد حساب کاربری خود شوید

عنوان ژورنال:

دوره   شماره 

صفحات  -

تاریخ انتشار 2012